Testing for Neglected Nonlinearity Using Regularized Artificial Neural Networks

The artificial neural network (ANN) test of Lee et al. (Journal of Econometrics 56, 269–290, 1993) uses the ability of the ANN activation functions in the hidden layer to detect neglected functional misspecification. As the estimation of the ANN model is often quite difficult, LWG suggested activate the ANN hidden units based on randomly drawn activation parameters. To be robust to the random activations, a large number of activations is desirable. This leads to a situation for which regularization of the dimensionality is needed by techniques such as principal component analysis (PCA), Lasso, Pretest, partial least squares (PLS), among others. However, some regularization methods can lead to selection bias in testing if the dimensionality reduction is conducted by supervising the relationship between the ANN hidden layer activations of inputs and the output variable. This paper demonstrates that while these supervised regularization methods such as Lasso, Pretest, PLS, may be useful for forecasting, they may not be used for testing because the supervised regularization would create the post-sample inference or post-selection inference (PoSI) problem. Our Monte Carlo simulation shows that the PoSI problem is especially severe with PLS and Pretest while it seems relatively mild or even negligible with Lasso. This paper also demonstrates that the use of unsupervised regularization does not lead to the PoSI problem. Lee et al. (Journal of Econometrics 56, 269–290, 1993) suggested a regularization by principal components, which is a unsupervised regularization. While the supervised regularizations may be useful in forecasting, regularization should not be supervised in inference.